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ABSTRACT 

A study examined the effects of statewide 
performance-based assessment (the Maryland School Performance 
Assessment Program (MSPAP)) on literacy instruction, administrative 
support provided to facilitate instructional change, and student and 
teacher affect in response to change in elementary classrooms. Ofc* the 
schools in Maryland identified by district-level administrators as 
making positive instructional changes in literacy in response to the 
MSPAP, five were selected as research sites due to their disparate 
approaches to school reform and the diversity of their populations. 
Information was gathered through semi-structured interviews with 
building administrators, reading specialists, and teachers in each of 
the five schools. Primary data sources were triangulated with 
secondary data sources (i.e», informal conversations with school 
personnel, state guidelines, school artifacts) and analyzed using 
constant-comparative methods. Findings revealed that the assessment 
program did have positive effects on instructional practice, 
administrative support, and participant affect. (Contains 36 
references and five tables of data. A time line of data collection 
procedures is attached.) (Author/RS) 
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About the National Reading Research Center 



The National Reading Research Center (NRRC) is 
funded by the Office of Educational Research and 
Improvement of the U.S. Department of Education to 
conduct research on reading and reading instruction. 
The NRRC is operated by a consortium of the Universi- 
ty of Georgia and the University of Maryland College 
Park in collaboration with researchers at several institu- 
tions nationwide. 

The NRRC's mission is to discover and document 
those conditions in homes, schools, and communities 
that encourage children to become skilled, enthusiastic, 
lifelong readers. NRRC researchers are committed to 
advancing the development of instructional programs 
sensitive to the cognitive, sociocultural, and motiva- 
tional factors that affect children's success in reading. 
NRRC researchers from a variety of disciplines conduct 
studies with teachers and students from widely diverse 
cultural and socioeconomic backgrounds in pre-kinder- 
garten through grade 12 classrooms. Research projects 
deal with the influence of family and family-school 
interactions on the development of literacy; the interac- 
tion of sociocultural factors and motivation to read; the 
impact of literature-based reading programs on reading 
achievement; the effects of reading strategies instruction 
on comprehension and critical thinking in literature, 
science, and history; the influence of innovative group 
participation structures on motivation and learning; the 
potential of computer technology enhance literacy; 
and the development of methods and standards for 
alternative literacy assessments. 

The NRRC is farther committed to the participation 
of teachers as full partners in its research. A better 
understanding of how teachers view the development of 
literacy, how they use knowledge from ■isearch, and 
how they approach change in the classroom is crucial to 
improving instruction. To further this understanding, 
the NRRC conducts school-based research in which 
teachers explore their own philosophical and pedagogi- 
cal orientations and trace their professional growth. 



Dissemination is an important feature of NRRC activi- 
ties. Information on NRRC research appears in several 
formats. Research Reports communicate the results of 
original research or synthesize the findings of several 
lines of inquiiy. They are written primarily for re- 
searchers studying various areas of reading and reading 
instruction. The Perspective Series presents a wide 
range of publications, from calls for research and 
commentary on research and practice to first-person 
accounts of experiences in schools. Instructional 
Resources include curriculum materials, instructional 
guides, and materials for professional growth, designed 
primarily for teachers. 

For more information about the NRRC's research 
projects and other activities, or to have your name 
added to the mailing list, please contact: 

Donna E. Alvermann, Co-Director 
National Reading Research Center 
318 Aderhold Hall 
University of Georgia 
Athens, GA 30602-7125 
(706) 542-3674 

John T. Guthrie, Co-Director 
National Reading Research Center 
2102 J. M. Patterson Building 
University of Maryland 
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Abstract. This investigation examined the effects of 
a statewide performance-based assessment (the 
MarylandSchoolPerformanceAssessment Program) 
on literacy instruction, administrative support 
provided to facilitate instructional change, and 
student and teacher affect in response to change in 
elementary classrooms. Of the schools in Maryland 
identified by district-level administrators as making 
positive instructional changes in literacy in response 
to the MS PAP, five were selected as research sites 
due to their disparate approaches to school reform 
and the diversity of their populations. Information 
was gathered through semi-structured interviews 
with building administrators, reading specialists, 
and teachers in each of the five schools. Primary 
data sources were triangulated with secondary data 
sources (i.e., informal conversations with school 
personnel, state guidelines, school artifacts) and 
analyzed using constant -comparative methods 
(Closer & Strauss, 1967; Strauss & Corbin, 1990). 
Findings revealed that the assessment program did 
have positive effects on instructional practice, 
administrative support, and participant affect. 



Current interest in using statewide assess- 
ments for high stakes, such as for curriculum 
reform or accountability, has met with opti- 
mism (Cooley, 1991;Popham, 1987;Popham, 
Cruse, Rankin, Sandifer, & Williams, 1985) as 
well as criticism (Bracey, 1987; Ellwein, 
Glass, & Smith, 1988; Shepard, 1991; Smith, 
1991; Smith & Rottenberg, 1991). Proponents 
of such measurement-driven instruction have 
suggested that well-developed assessments may 
be the most cost-effective way of improving 
the quality of education (Popham, 1987). 
Some critics, however, have suggested that 
such practice may lead to a fragmented, nar- 
row, and trivialized curriculum that is deflect- 
ed from its intended purposes (Bracey, 1987). 
Others have suggested that measurement-driven 
instruction implies a behavioristic view of 
learning, which suggests that basic skills 
should be taught and mastered before attempt- 
ing to teach higher-order thinking (Shepard, 



9 



2 



Almasi, Afflerbach, Guthrie and Schafer 



1991). Thus, if assessments are to drive in- 
struction positively, they must include tests of 
higher-order thinking skills. 

With increased attention being given to 
developing national standards for educational 
reform and in light of the criticism of tradition- 
al, norm-referenced standardized tests (Freder- 
iksen, 1984; Hiebert, Valencia, & Afflerbach, 
1994; Smith, 1991; Smith & Rottenberg, 
1991), enthusiasm has mounted for developing 
alternative measures aimed at assessing the 
higher-order cognitive abilities associated with 
such standards (Nickerson, 1989). Thus, con- 
certed efforts have been made to develop 
assessments that are more authentic and less 
susceptible to the criticisms of more traditional, 
high-stakes measures . Authentic assessment has 
been defined as "assessment activities that 
represent literacy behavior of the community 
and workplace, and that reflect the actual 
learning and instructional activities of the 
classroom and out-of-school worlds" (Hiebert, 
Valencia, & Afflerbach, 1994, p. 11). The 
promise of this form, of assessment for high- 
stakes purposes may be that if tests truly have 
the potential to influence the instruction that 
teachers give their students, then an improved 
assessment that reflects real-world learning and 
higher-order cognitive processing would lead to 
improvements in instructional practice. 

The Maryland School Performance 
Assessment Program 

The Maryland School Performance Assess- 
ment Program (MSPAP) is employing perfor- 
mance assessment as a means of holding 
schools accountable for students' learning and 
of reforming the curriculum (Kapinus, Collier, 



& Kruglanski, 1994; Maryland State Depart- 
ment of Education, 1991). The 
MSPAP— which is among the newer forms of 
high-stakes , statewide assessment— was derived 
as a response to a state mandate for public ac- 
countability of schools, school systems, and of 
the state for its role in student performance. 
This mandate for change resulted in the forma- 
tion of a task force commissioned by the gov- 
ernor to develop a set of suggestions for 
change. This task force produced a document 
known as "The Report of the Governor's 
Commission on School Performance" (Sond- 
heim et al., 1989) that became the blueprint for 
change within the state. Additionally, the 
Maryland State Department of Education 
(MSDE) was entrusted with the establishment 
of a statewide school improvement program 
that would achieve such accountability (Sond- 
heim et al., 1989). The Maryland School 
Performance Program (MSPP) was thus estab- 
lished, and the MSPAP became a part of that 
school improvement initiative. 

The purpose of the present investigation 
was to determine the impact of the literacy 
portions of the MSPAP on instructional prac- 
tice. Thus, a description of the literacy out- 
comes of the MSPAP, implementation proce- 
dures, and literacy assessment activities follow. 
Each component will also be assessed in terms 
of how its test developers have attempted to 
provide for concerns about validity (Freder- 
iksen & Collins, 1989; Linn, Baker, & Dun- 
bar, 1991; Messick, 1994). 

MSPAP Reading Outcomes 

The MSPAP focuses on educational out- 
comes that Maryland adopted in 1990 and 
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hoped to attain by the year 2000. The reading 
outcomes developed for the MSPAP include: 
(1) demonstrating positive attitudes toward 
reading; (2) constructing, extending, and 
examining meaning when reading for literary 
experience; (3) constructing, extending, and 
examining meaning when reading for infor- 
mation; (4) constructing, extending, and exam- 
ining meaning when reading to perform a task; 
and (5) demonstrating awareness of strategic 
behaviors and knowledge about reading (Kapi- 
nus, Collier, & Kruglanski, 1994; Maryland 
State Department of Education, 1991). The 
adoption of such outcomes before the assess- 
ment was designed ensures that the perfor- 
mance assessment is construct driven rather 
than task driven (Messick, 1994; Popham, 
1987). 

MSPAP Implementation Procedures 

The MSPAP has been administered to every 
third-, fifth-, and eighth-grade student in the 
state of Maryland since 1991. In 1991, the 
MSPAP only measured student performance in 
reading/language arts and mathematics. The 
following year, social studies and science were 
added to the assessment. All tasks on the 
MSPAP are open-ended in nature. Thus, the 
tasks often require considerable time— up to 90 
minutes in some cases— to complete, reflecting 
the test developers' bias toward depth of pro- 
cessing over breadth of coverage. The issue 
that arises then is one of construct general- 
izability (Linn, Baker & Dunbar, 1991; Mess- 
ick, 1994). That is, if long, open ended tasks 
are used, students cannot be assessed on every 
outcome in a given domain, and scores would 
not accurately reflect students' knowledge. 



This problem was circumvented by using 
matrix sampling; no student takes the entire 
assessment, but the entire assessment is admin- 
istered to each school. Thus, a picture of how 
we?l the school addresses all of the learning 
outcomes is reported but individual scores are 
not (Kapinus, Collier, & Kruglanski, 1994). 

Reading Assessment Activities 

The reading portions of the MSPAP are 
unique in several ways. The MSPAP uses 
authentic, unedited texts that include stories, 
poems, articles, directions, and chapters from 
trade books. Since the addition of social studies 
and science in 1992, reading materials also 
include maps, charts, and directions for hands- 
on science lessons that are integrated with tasks 
centered around a common theme. Throughout 
the MSPAP, students are also given choices: 
they can 'select from three or four stories to 
read; they may have their choice of writing 
tasks in response to a given selection (e.g., 
writing a story, poem, or play); or they may 
have choice in response formats (e.g. , drawing 
or writing) for a question (Kapinus, Collier, & 
Kruglanski, 1994). The MSPAP also measures 
students' writing proficiency through longer 
activities that permit students to use the entire 
writing process: prewriting, drafting, peer 
response, revision, and editing. Thus, the 
assessment tasks are in line with current views 
of what good instruction in literacy encompass- 
es (Wixson, 1994). 

In an attempt to reduce bias and to address 
issues of fairness (Linn, Baker, & Dunbar, 
1991), the developers of the MSPAP began all 
tasks with preassessment activities designed to 
fill gaps in students' background experiences 
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(Kapinus, Collier, & Kruglanski, 1994), Many 
of these activities provide for student coopera- 
tion and collaboration in small groups. 

Purposes of this Investigation 

While ome educational measurement 
special . contended that specialized 

validity criteria shou^ be established that are 
sensitive to the expectations of performance- 
based assessment (Frederiksen & Collins, 
1989; Linn, Baker, & Dunbar, 1991), others 
contend that these specialized criteria for 
performance-based assessments are consistent 
with general standards of validity as described 
by Messick (1989) and further suggest that 
performance assessments should be held to the 
same validity criteria as other assessments 
(Messick, 1994; Moss, 1992). All agree, 
however, that validation of an instrument 
requires creating a basis for score interpretation 
and use (Linn, Baker, & Dunbar, 1991; Mess- 
ick, 1989; 1994). The concern is that perfor- 
mance assessment may lead to enhanced in- 
structional practice (i.e., consequential vali- 
dity), and evidence may be overlooked as a 
basis for determining consequential validity. 
Therefore, we need to know the intended and 
unintended effects of performance assessment 
on the variables targeted for change, such as 
the way teachers think about instruction and the 
goals of instruction that are set as a result of 
performance assessment in general (Linn, 
Baker, & Dunbar, 1991; Messick, 1989, 1994; 
Tittle, 1989) and the MSPAP in particular 
(Wixson, 1994). Although positive signs of the 
effects of the MSPAP on teachers' professional 
growth have been observed (Kapinus, Collier, 
& Kruglanski, 1994), the consequences of the 



MSPAP on instructional practice have yet to be 
investigated systematically. 

Thus, the present investigation examined 
the effects of a statewide, performance-based 
assessment (the MSPAP) on three variables: 
literacy instruction, administrative support for 
changes in instructional methods, and student 
and teacher affect resulting from changes. In 
particular, we were interested in exemplars of 
positive school reactions to the MSPAP. 
Therefore, this investigation does not focus 
directly on any negative school outcomes (for 
such information, see Afflerbach, Almasi, 
Guthrie, & Schafer, 1994). 

METHOD 

Selection of Sites 

Sites were chosen during an exploratory re- 
search phase prior to the present study (for 
further information, see Guthrie, Almasi, 
Schafer, & Afflerbach, 1994). The first step of 
the selection process involved interviewing a 
representative from each of the 24 school 
districts in Maryland to determine the types of 
innovations occurring within their districts as 
a result of the MSPAP and to identify schools 
within each district that were exemplars of 
positive change. (Representatives from 21 of 
the 24 school districts in Maryland were actu- 
ally contacted.) These representatives were 
employed by their respective Board of Educa- 
tion offices and held positions such as: Su- 
pervisor of Reading (n = 5), Supervisor of 
Elementary Education (n = 5), Supervisor of 
Instruction (n = 3), Supervisor of Language 
Arts (n = 3), Director of Curriculum (n = 2), 
and (one each) Reading Specialist, Principal, 
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and Coordinator of Chapter 1 and Testing. 
These individuals were sent letters stating the 
purpose of the research project, describing 
their role in the project, and including the 
questions that would be asked during the inter- 
view. 

The structured interview consisted of 13 
items that were drafted by a team of four 
researchers. The primary goal of the interview 
was to identify one or two schools within the 
district that were initiating changes in response 
to the MSPAP. A secondary goal was to deter- 
mine what types of changes were occurring at 
the county level in response to the MSPAP. 
Interviews were conducted by telephone, and 
each representative was asked the entire set of 
questions. Responses were recorded as fully as 
possible by hand. 

Forty-two elementary schools were nomi- 
nated by the 21 county representatives. De- 
mographic data regarding the characteristics of 
each school and the types of innovations being 
initiated in each school were recorded. Types 
of innovations were determined using constant- 
comparative methods; innovations were in- 
spected initially to gain a holistic sense of the 
data. 

Five categories of innovations emerged: 

(1) integrated curriculum (n = 11, 26.2%), 

(2) literature-based instruction (/* = 6, 14.3%), 

(3) ' principal as strong leader in initiating 
change (n = 6, 14.3%), (4) strategic instruc- 
tion (n = 5, 11.9%), and (5) emergent literacy 
programs (n = 3, 7. 1 %). The innovative prac- 
tices from 1 1 of the schools (26.2%) did not fit 
into this categorization system and did not 
warrant the creation of additional categories 
because each innovation was unique. Innova- 
tions in the 1 1 other schools included practices 



such as scoring holistically on assessments, 
making grouping changes, employing an un- 
graded curriculum, using computers, conduct- 
ing summer reading programs and community- 
based reading programs, writing grants, and 
attending to the needs of English as a Second 
Language (ESL) students. These innovations were 
often features of— and subordinate to— the larg- 
er programs operating in the other schools and 
thus were not included as separate categories. 

Table 1 displays the demographic data from 
the 42 nominated schools with respect to racial 
composition, socioeconomic status (SES), size of 
school, whether the school received Chapter- 1 
funding, and type of innovation initiated. 

Five schools were then selected for partic- 
ipation in the investigation, based upon type of 
innovation and demographic characteristics. 
Schools were selected by a team of researchers 
familiar with the results of the exploratory 
study. The 31 schools representing the five 
types of innovations were separated by catego- 
ry. Within each category, schools were priori- 
tized by demographics. We were especially 
concerned with attaining information on how 
schools with at-risk populations were initiating 
change, so, within each category, schools with 
large minority populations, urban populations, 
lower SES, or Chapter- 1 funding were given 
priority. Of the five schools that were selected, 
two served predominantly minority popula- 
tions, and three served predominantly white 
populations; two were in communities with low 
SES, one with middle SES, one with middle/ 
high SES, and one school served a high-SES 
community. Two schools received Chapter- 1 
funding; two had student populations over 800; 
two had between 401 and 600 students; and 
one school had 300 students. 
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Table !• Demographic Data for Schools Nominated. 



fhirQPtprictfp 
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Racial Composition* 






Predominantly white 


35 


83.0 


1V1IACD 


3 


7.1 


Predominantly minority 


4 


9.5 


Socioeconomic Status* 






Low 


7 


16. 7 


Low/Middle 


7 


16.7 


Middle 


16 


38.1 


Middle/High 


3 


7.1 


High 


3 


7.1 


Range of 3ES 


6 


14.3 


v.napier 1 runciing 






i es 


Z4 






1 Q 




Number of Students** 






200-400 


11 


29.7 


401-600 


19 


51.4 


601-800 


5 


13.5 


>800 


2 


5.4 


Type of Innovation* 






Integrated curriculum 


11 


26.2 


Literature-based curriculum 


6 


14.3 


Principal as strong leader 


6 


14.3 


Strategy-based instruction 


5 


11.9 


Emergent literacy program 


3 


7.1 


Other 


10 


23.8 



♦percentages based on data from 42 schools 
♦♦percentages based on data from 37 schools 



Sites and Participants 

School A 

The community. School A is one of 1 18 ele- 
mentary schools in an urban public school 
system. The school is situated amidst hundreds 



of government-subsidized public housing units, 
several hundred of which are vacant. The 
residents of the community are almost exclu- 
sively African American and have a low SES 
($4,255 per-pupil expenditure; 68% participate 
in the frt^/reduced lunch program). Table 2 
displays information regarding per-pupil ex- 



NATIONAL READING RESEARCH CENTER, READING RESEARCH REPORT NO. 32 



Instructional Change 



7 



Table 2, Demographic O aracteristics of School Sites Selected for Participation. 









School 






Characteristic 


A 


B 


C 


D 


E 


Per-Pupil 
Expenditure 


$4,255.00 


$5,549.00 


$4,396.00 


$4,191.00 


$6,629.00 


Participation in 
free/reduced 
lunch program 


68.0% 


5.0% 


26.0% 


3.0% 


60.4% 


School population 


296 


500 


598 


804 


813 


Caucasian 


0.0% 


97.0% 


72.0% 


92.0% 


17.1% 


African American 


99.7% 


3.0% 


28.0% 


8.0% 


29.6% 


Latino 


0.03% 


0.0% 


0.0% 


0.0% 


44.8% 


Other 


0.0% • 


0.0% 


0.0% 


0.0% 


8.4% 



♦Information obtained from Maryland School Performance Program Office (1990) 



penditure and the percentage of students partic- 
ipating in the free/reduced lunch program in 
each school. This information, combined with 
interviews with school personnel, was used to 
determine the SES of each school. 

The school, student body, and staff. 
School A is an aging, two-story brick building. 
Windows of cloudy Plexiglas and locked iron 
doors guard its exterior. Rows of metal lockers • 
line the walls of the main hallway. The media 
center is located on the second floor of the 
building, away from the flow of traffic; the 
interior of the media center contains many 
round wooden tables, some chairs, and large 
bookshelves with ample room for more books. 

The school's population consists of 296 
pre-kindergarten through fifth-grade students, 
the majority of which are African American 
(see Table 2). At the time of the interviews 



(February, 1993), the school was scheduled for 
closing due to declining enrollment. School A 
was selected primarily because it represents an 
inner-city school attempting change amidst a 
myriad of typical urban problems. During 
interviews with school officials we learned that 
many of the students came from low-SES 
homes with undesirable living conditions; 
several were homeless. The school receives 
Chapter- 1 funding. 

The school's curricular changes center 
primarily around its emergent literacy pro- 
gram. The person who was the catalyst for 
much of the change— the school's speech/ 
language therapist (the school does not have a 
reading specialist)— was unavailable for the 
interview. Arrangements for the interviews 
with faculty members were made by the build- 
ing administrator, who held a doctoral degree 
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in educational administration and who was 
interested in reading education. The interviews 
were confined to one in-service day at the 
principal's direction. Using an in-service day 
avoided encroaching on teachers' normal 
planning times and probably meant they were 
more willing to share information. 

School B 

The community. School B is located in an 
affluent suburban area and lies at the center of 
the Baltimore- Washington corridor. The sur- 
rounding neighborhoods consist of housing 
developments containing large, single-family 
dwellings with well-manicured lawns. Parents 
are actively involved in all school events, and 
95-98% of the parents attend report-card con- 
ferences. Many of the parents are highly edu- 
cated and are vocal with respect to the changes 
that are being made in the school's curriculum. 
As indicated by Table 2, a high per-pupil 
expenditure ($5,549) and a lower percentage of 
students participating in the free/reduced lunch 
program (5%) attests to the high SES of the 
community surrounding School B. 

The school, student body, and staff. 
School B is a modern, brick building with a few 
tall, tinted windows. It is one of 28 elementary 
schools in the district. The school sits one- 
quarter mile off the road, surrounded by woods 
and an ample playground containing bright blue 
and yellow play-sets . Upon entering the school , 
one is greeted by bright red carpeting; poster 
depicting large yellow pencils boldly proclaim, 
"School B: Sharp Minds at Work." The media 
center is located at the center of the school and 
has no doors; one must pass through the media 



center to get to the other side of the school 
Dozens of student-made dioramas depicting 
scenes from children's literature decorate the 
tops of the bookshelves, which are brimming 
with a multitude of selections. 

The 500 students of School B are predomi- 
nantly Caucasian (see Table 2). School B was 
selected as a school with a principal who was 
a strong leader in instructional change. This 
fact was apparent in the comments of county 
officials and faculty members, as well as in the 
actions and professional activities of the princi- 
pal. She had been a reading specialist for nine 
years, had presented workshops on effecting 
school change to other administrators across 
the state, was very active in professional orga- 
nizations such as the Association for Supervi- 
sion and Curriculum Development (ASCD), 
and held a doctoral degree in educational 
administration. 

School C 

The community. School C is located in 
southern Maryland in one of five Maryland coun- 
ties that make up the Washington-Maryland- 
Virginia metropolitan area. The school is 
located in a suburban, yet slightly rural, com- 
munity. Modest, single-family dwellings are, 
located nearby but are not visible from the 
school. Residents of the community are em- 
ployed in a variety of occupations and range 
from military personnel to educators to execu- 
tives. Most residents, however, are of medium 
SES (see Table 2), and many of the students 
come from single-parent homes. Parents and 
community members are eager to be involved 
in school events. 
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The school student body, and staff. 
School C is a brand new public school that 
opened for the 1992-93 school year. Modern 
in appearance with state-of-the-art facilities, it 
is one of 16 elementary schools in the county 
system. School C is located on a huge tract of 
open, flat land, and the only visible structure 
nearby is the high school, which is one-quarter 
mile in front of the school. The 598 students 
(kindergarten through grade five) who attend 
the school are predominantly white (72%) and 
are of medium SES (see Table 2). The school 
also houses special education students with 
emotional adjustment problems. 

The principal had her master's degree in 
educational administration and was passionate 
about creating a staff that had a vision for the 
fixture of the school and that emphasized com- 
munication. School C was selected because it 
featured an integrated curriculum. Literature- 
based instruction was integrated throughout the 
content areas as well as portfolio assessment 
programs. 

School D 

The community. School D is located in 
northeastern Maryland and is part of the Balti- 
more metropolitan area. It is located in an 
upper-middle-class suburban neighborhood 
consisting of single-family homes. Parents in 
the community hold a variety of occupations 
ranging from physicians to truck drivers; 
however, most hold white-collar, executive 
positions in the community. Parents are inter- 
ested in their children's education, as was 
shown by the support of the Parent Teacher 



Association in providing funds for additional 
items. 

The school, student body, and staff. 
School D was built in 1990 as a pilot site for 
a curriculum that was eniirely literature-based. 
The school enrolls 804 students (kindergarten 
through grade five), 92% of whom are white 
and are primarily upper-middle SES (see Table 
2). The school is an expansive brick building. 
The spaciousness of the surrounding property 
and within the school itself is striking. 

The principal had his master's degree in 
educational administration and was interested 
in making a literature-based program succeed. 
In fact, the school was selected because its 
curriculum was entirely literature-based. In 
preparing for the opening of the new school in 
1990, faculty members from 23 elementary 
schools in the county were interviewed. A 
diverse staff was chosen, including some 
members who came from very traditional, 
basal reader programs featuring homogeneous 
grouping and structured activities. Some staff 
members came from literature-based reading 
programs in which trade books and flexible 
grouping patterns were used. This diverse staff 
had to quickly congeal and adapt to a new 
school where no basal readers were purchased 
and where trade books and thematic teaching 
would be expected. Also, the school opened 
without a functional media center. In spite of 
this difficulty and the diversity in teaching 
experiences among staff members, county and 
school officials said that faculty members 
generally accepted the literature-based curric- 
ulum. This intriguing factor was another rea- 
son for selecting School D as a research site. 
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School E 

The community. School E rests on the 
border of the Washington D.C. metropolitan 
area and is located in a suburban neighborhood 
consisting of single-family dwellings and 
mid-rise apartment buildings. School E is one 
of 118 elementary schools in a county whose 
per-pupil expenditure ($6,629) is one of the 
highest in the state of Maryland (see Table 2). 
Small commercial centers occupy the areas on 
the periphery of the residential communities. 
The residents served by this school are primar- 
ily Hispanic and African American and have a 
low SES; many are immigrants with little 
education and limited English proficiency. 
Residents are often employed in unskilled labor 
occupations and live in tiny homes with no 
furniture and no printed material. Many will 
move after a short time, resulting in a very 
transient population. 

The school, student body, and staff. 
School E is a large, two-story, modern build- 
ing that houses 813 students from preschool 
through sixth grade. The school was funded as 
an International Magnet School, meaning that 
students of culturally diverse backgrounds and 
students who are non-native English speakers 
attend the school. They come from many 
communities around the county. School enroll- 
ment data attest to the diversity of the student 
population and the low SES (see Table 2). The 
school curriculum includes a half-day Spanish 
Immersion Program in which 32.5% of the 
students participate. 

The school made innovations related to 
strategic instruction as well as to serving its 
multicultural population. Because the school is 



an International Magnet serving a population 
quite distinct from the remainder of the county, 
it is supported by several federally-funded 
programs (Head Start, Chapter 1) and county- 
funded resources (e.g., curriculum coordi- 
nator, parent coordinator, ESL teachers) that 
are not available to other schools in the county. 
In hiring faculty members, the principal con- 
siders several factors, including candidates' 
energy level, whether or not they can manage 
a classroom effectively, aad whether they have 
experience working amidst cultural diversity 
(e.g., in the Peace Corps or teaching abroad) 
or have extensive travel experience so as to be 
aware and tolerant of cultural differences. The 
principal is a former reading specialist who 
reads extensively about educational research on 
policy as well as practice and who holds a 
doctoral degree in educational administration. 

Interview Protocol 

The eight-item semi-structured interview 
used during the school-based interviews was 
developed by a team of four researchers famil- 
iar with educational measurement, assessment, 
classroom procedures, and the MSPAP itself. 
Many of the items were similar to those asked 
of county representatives during the explorato- 
ry research phase. Before the actual interviews, 
questions were piloted with teachers and ad- 
ministrators who were not in the sample of 
interest. Alterations based on the information 
collected during pilot interviews guided the 
construction of questions. 

Three types of information were of primary 
interest: (1) the goals of the MSPAP and its 
impact on literacy instruction, (2) the manner 
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in which changes/innovations were facilitated, 
and (3) the challenges that were encountered in 
attempting to make changes. The focus of the 
present investigation centers on the effect of the 
MSPAP on literacy instruction and the ways in 
which such changes were facilitated. (See 
Afflerbach, Almasi, Guthrie, and Schafer 
[1994] for reports of challenges encountered as 
such changes were implemented.) 

The interview consisted of the following 
eight questions: 

1. What is the goal of the Maryland School 
Performance Assessment Program (MSP- 
AP)? 

2 . What do you think the MSPAP measures in 
terms of reading performance? 

3. How would you describe the instructional 
changes that have occurred in your school 
in relation to the Reading and Language 
Arts portions of the MSPAP? 

4. What impact has the MSPAP had on your 
school that we have not already discussed? 

5. Where did you get ideas for these changes? 

6. What resources do you have for making 
these changes? 

7 . What materials do you use to assist students 
as they prepare for the Reading/Language 
Arts portions of the MSPAP? 

8. What barriers (if any) did you face in trying 
to make these changes? 



Data Collection and Analysis 

Data Collection 

During the interview procedures, the re- 
searcher functioned primarily as an inquisitor 
and as a facilitator of group process, asking 
only the interview questions and probing when 
answers were unclear, vague, or brief. All 
interviews were audiotaped, and brief field 
notes were taken at each of the research sites; 
field notes were fleshed out later. Thus, prima- 
ry data consisted of field notes and audiotaped 
interviews collected during one- or two-day 
sessions in each school throughout a three- 
month period (see Appendix). All interviews 
with school personnel were arranged by the 
building administrator prior to the day data 
were collected. Interviews were conducted with 
building administrators (e.g., principals, assis- 
tant principals), curriculum specialists (e.g., 
reading specialists, curriculum coordinators, 
supervisors), and teachers (third-grade teachers 
and fifth-grade teachers). These individuals 
were interviewed because each had diverse and 
expert knowledge about the construction and 
administration of the MSPAP and about how 
instructional change was achieved in their 
school. All interviews were conducted in 
private rooms and were recorded on audiotapes 
that were later transcribed in their entirety. 

Data from school administrators and curric- 
ulum specialists were collected individually; 
but teams of third-grade teachers and teams of 
fifth-grade teachers were interviewed in small- 
group settings so that teachers could converse 
openly. This procedure was piloted during the 
preliminary research phase and was found to 
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be effective at reducing teachers' anxiety about 
the interview process. Also, teachers often 
embellished the comments of their colleagues 
to create a richer picture of the curricular 
changes they were describing. 

Data Analysis 

Primary data sources consisting of trans- 
cribed interviews and field notes were trian- 
gulated with secondary sources that included: 
(1) informal conversations with school officials 
(county supervisors and building administra- 
tors); (2) state documents (Maryland School 
Performance Program Results); and (3) school 
artifacts, including school enrollment data and 
instructional materials used by teachers in the 
schools. The data were analyzed using the 
constant-comparative method (Glaser & 
Strauss, 1967; Strauss & Corbin, 1990). The 
analysis procedure consisted of listening to the 
audiotapes and rereading field notes and tran- 
scripts of interviews to gain a holistic sense of 
the data. Three copies of the transcripts from 
each interview (n = 22) had been made so that 
highlighting and coding could be done. Tran- 
scripts were then read thoroughly, and portions 
that addressed each of the interview questions 
were highlighted and coded. Highlighted por- 
tions representing information pertinent to each 
question were then copied onto data sheets for 
each interview. The intact copies of the tran- 
scripts, as well as the original audiotapes were 
often referred to throughout the process. 

Initial categorizations were established with 
respect to each of the interview questions; these 
were supplemented by assertions regarding 
linkages between the information gleaned from 
each school as well as information gleaned 



from all five schools. (Two researchers re- 
viewed the data several times.) This process 
helped reduce the problem of premature accep- 
tance of categorization systems. During the 
categorization process, properties emerged that 
added to the conceptual density of the catego- 
ries (Strauss & Corbin, 1990). Once the cate- 
gories seemed fully saturated, the data were 
revisited in order to test the categories, proper- 
ties, and dimensions. This analysis allowed 
conclusions to be drawn both within the five 
sites and across the five research sites regard- 
ing the effect of the MSPAP on instruction, the 
means of achieving instructional change, and 
any barriers to change that might exist. 

RESULTS AND DISCUSSION 

The major implication of the data collection 
and analysis is that the MSPAP did have a 
direct influence on the types of instructional 
changes taking place in all five schools. The 
categories that supported this assertion were 
found in response to questions related to (1) the 
types of curricular changes occurring in these 
schools and (2) how change was facilitated in 
these schools. A third category emerged that 
was not directly linked to a specific question 
but which emerged as a result of consistent re- 
porting across all schools, primarily as respon- 
dents were addressing issues related to ques- 
tions three and four of the interview protocol. 
This final category, then, related to (3) the 
effects instructional changes have on the stu- 
dents who experienced them and the teachers 
who were responsible for implementing those 
changes. Thus, the categories include (1) rea- 
lignment of curriculum, materials, and class- 
room cultures to match features of the MSPAP, 
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Table 3. Properties and Dimensions of Instructional Change in Schools 



Properties 
(Dimensions) 


A 


B 


Schools 
C 


D 


E 


Level of congruence 


low 


med-high • 


high 


high 


high 


Speed of implementation 


slow 


gradual 


undetermined 


grad/abrupt 


gradual 


Degree of change 


varied 


little 


varied 


varied 


varied 



(2) professional support at the district/county 
level and school level for teachers to facilitate 
change, and (3) affective changes in students 
and teachers. Evidence in the form of interview 
comments and supporting documents will be 
presented to illustrate how each of the three 
categories supports the major assertion. Since 
each school attempted change differently, 
however, a description of how change was 
achieved at each school will be provided before 
the discussion of each category. 

Achieving Change 

Interviews with school personnel in each of 
the five schools provided consensus that in- 
structional practice in their schools was affect- 
ed by the MSPAP. This change, however, did 
not occur in the same way in all schools. 
Closer examination of the interview data re- 
vealed that three properties of instructional 
change existed in each school: (1) the level of 
congruence between the MSPAP and instruc- 
tional practice and curriculum, (2) the speed 
with which change had been implemented, and 

(3) the degree of change experienced by the 



faculty. Based on interview comments, each 
property was then dimensionalized. Table 3 
illustrates the ranges of variation that were 
possible within each property, as well as each 
school's profile. 

School A 

According to school personnel, School A 
produced minimal amounts of instructional 
change after the first administration of the 
MSPAP. The MSPAP and the school's curric- 
ulum appear to have been somewhat aligned. 
However, congruence varied from classroom 
to classroom. As the principal noted: 

There's an awareness that we've got to 
make some changes, but to be perfectly 
honest the school is judged still by the 
Comprehensive Test of Basic Skills (CTBS), 
which is the traditional multiple-choice test, 
and I've even told my staff members, 
"We've got to make some changes for the 
MSPAP, but remember when the results are 
published in the newspapers they are look- 
ing at the CTBS— and that's important. So 
1 guess we're more at an awareness level. 
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The principal's implication that students' 
performances on the CTBS were more im- 
portant than the need to produce change based 
on the MSPAP was substantiated during visits 
to classrooms. Much shelf space was taken up 
by "Scoring High" booklets (CTBS test prepa- 
ration booklets). When questioned about their 
use of the booklets during class time, teachers 
said that students would be doing a little bit of 
practice in the booklets every day until the 
CTBS was administered in April. Since these 
interviews occurred on January 25th, (see 
Appendix), a good bit of time was probably 
spent in preparation for the more traditional, 
standardized assessment. 

The school did, however, experience some 
change as a result of the MSPAP, although 
change was slow. The principal noted that, 
"Some [teachers] are making a lot of changes 
[as a result of the MSPAP]; some are making 
minimal changes; maybe one or two haven't 
made any changes." This variation among 
teachers was apparent during interviews with 
the third- and fifth-grade teachers. The teacher 
responsible for teaching the gifted and talented 
pull-out program was interviewed with the 
third-grade teachers. She explained that she 
creates thematic units with the trade books she 
uses. She also noted that she uses Reader's and 
Writer's Workshops in her classroom. Inter- 
views with the fifth-grade teachers revealed 
differences in the curriculum for the gifted and 
talented students and for others. 

School B 

The level of congruence between the curric- 
ulum at School B and the MSPAP ranged from 
medium to high; change occurred gradually 



over a period of years. The reading specialist 
had noticed several disturbing aspects of stu- 
dents' literacy behaviors and had begun to sug- 
gest that changes occur just prior to MSPAP, 
according to the principal: 

[The reading specialist] said, before all of 
this [the MSPAP] ever came out four years 
ago [that the] kids were basically not liking 
to write. They could read, but when it got 
to the writing, or responding to tasks, or in- 
corporating higher level thinking skills, they 
weren't doing it, and you couldn't measure 
it. We were having a hard time, and dittoes 
were overkill. We had to get rid of them. 
Kids were tired of reading out of basals. It 
was turning them off. 

The suggested changes were congruent with the 
MSPAP, as is reflected by the reading special- 
ist's comment: 

My feeling is we are finally headed in the 
same direction. We're moving forward 
together— the curriculum reflects the state 
outcomes— for once we have an instrument 
that reflects what we're doing in the class- 
room. 

This statement, coming from someone who 
works with the entire faculty, suggests that the 
curriculum as a whole is aligned with the 
outcomes of the MSPAP. The comments of the 
fifth-grade teachers support the reading special- 
ist's view: 

Somebody said [to me], "Are you going to 
prepare your kids [for the MSPAP]?" and 
we looked at them and we said, "We don't 
need to— our kids have been prepared since 
September! " This is the first year that when 
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our kids sit down and take the test it's like 
they've done this all year long. 

Thus, the changes in School B began about 
four years ago (1989) and have evolved in 
varied ways over the years. 

School C 

The high level of congrueuce between the 
curriculum at School C and the MSPAP is 
attributed to the fact that it was a brand new 
school. The principal noted that she had had an 
opportunity to meet with the central office sup- 
port staff to plan her school's curriculum so 
there would be a match between her ideals, the 
county's ideals, and the state's ideals: 

We had the opportunity . . . to . . . take 
some risks and develop some of the things 
we wanted to start. So, I had an advantage 
over some other schools because I was 
forming a new school; and we bought into 
this [the outcomes that the MSPAP is based 
upon]— from the central office support staff, 
from research, from information across the 
state— all of these things were guiding us 
too, but in our hearts, as educators, we 
knew this was the way to go. 

This high level of congruence is apparent in the 
remarks of a fifth-grade teacher: 

When we take the Maryland test [the MSP- 
AP], I don't even think they're [the stu- 
dents] going to realize it's a test because it's 
our normal day-to-day thing going on— 
here's your group, your materials, solve 
this problem. The only thing different is it's 
going to be collected and count for some- 
thing somewhere, and they don't get it back. 



Because the school was new, it is difficult 
to know how fast changes were implemented. 
However, the principal said the degree of 
change varied from teacher-to-teacher, and that 
all teachers wer* experiencing some change 
because they were creating a new curriculum 
that in many ways was aligned with MSPAP. 

School D 

School D also is a relatively new school. Its 
experiences are unique in that, at the time the 
MSPAP was coming out, the school was being 
built. This meant that teachers with varied 
backgrounds from around the county had to 
meld into a cohesive unit. All of the individu- 
als interviewed, however, agreed that their 
curriculum was intended to be well-aligned 
with the MSPAP. The comments of the third- 
grade teachers, who had brought portions of 
their county language arts guide with them, 
reflect this congruence: 

Most of the ideas [for change s ome from 
the county level where we have oeen given 
guidance from the committee developing an 
Integrated Language Arts Guide. This is 
directly in line with MSPAP. In fact, I have 
pieces of it here. On Tuesday, we had a 
meeting on Program Outcomes to make 
sure that we were in line with the state 
[outcomes] . We [the school] are even trying 
to get in line with national, state, and coun- 
ty outcomes. 

Apparently, change is occurring among all 
teachers, but the speed and degree to which 
this has happened varies, most likely because 
reforms coincided with the school's opening. 
Changes taking place in School D were ap- 
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parently complicated by the demand that people 
from varied backgrounds make curricular 
change. The principal described this delicate 
situation best: 

When we opened, we pulled staff members 
from 23 different schools. Some came from 
a very strict, structured basal program 
where you had 60 minutes for reading, and 
then reading is over. Then you clear your 
desk because it is time for reading compre- 
hension. It [the basal program] even told 
when to close your book and do your spell- 
ing. Others came from schools, and my 
previous school was one of them, where we 
had a literature-based program, and we had 
trade books, and we were using a modified 
grouping plan across grade levels. There 
was a fairly wide range of change necessary 
when this sfcbool opened so it is hard to say 
what change occurred in the last three 
years. For some people there were quantum 
leaps in change because they had to go cold 
turkey on the dittoes, and workbooks were 
no longer ordered .... We have had to 
provide varying amounts of support. 

Some of the teachers felt that because School D 
was intended to be a model school— one re- 
flecting the latest curricular innovations that 
were based in part on the MSPAP— that chang- 
es were being thrust upon them without ade- 
quate preparation. The teachers' frustrations 
are evident in this teacher's comments: 

This school was created about the time that 
MSPAP was created, and though we all 
were coming from different places where 
we were doing different things, we weren't 
really making a transition as a school. We 



had pretty much started from ground zero, 
and as we say it, as MSPAP was being 
thought up, the school's new curriculum 
was being thought up, and we were being 
swept along. So I know I still feel like we 
are flying in an airplane, and we are all still 
trying to put the wings together. 

The same teacher also felt as if the changes that 
were being made were abrupt. He had come 
from a basal school and in one summer had to 
make changes: 

[Making changes with a new school] is a lot 
different than I think [it would be] if you 
came from a school where they were work- 
ing in the basal five years ago, and now 
were working toward literature and making 
a gradual transition. We made a sudden 
change. 

The reading specialist's comments also reflect 
the frustration that went with making changes; 
however, she notes that although change may 
have been abrupt initially for some teachers, 
they gradually grew into their new roles: 

A lot of different things are happening in 
this school right now, and I see people 
much more at ease with it now .... I think 
the thing that is interesting to me is to watch 
the change ... at the beginning for a lot of 
people it was a major shift in the way they 
think about reading, writing, and children. 
As they've gotten into it and had this expe- 
rience, as well as listening to other people, 
and reading, they are growing, and you can 
see that reflected in the work they are doing 
because it shows in the progress of the 
children. 
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Although genuine change seems to have oc- 
curred within School D, it seems that the speed 
of implementation and the degree of change 
varied. 

School E 

School E has made definite changes in a 
thoughtful, gradual manner. The former curric- 
ulum coordinator at the school had worked 
closely with the Maryland State Department of 
Education as the MSPAP was being developed 
and was aware of the changes that were coming 
before the MSPAP was first administered. The 
principal noted that many of the changes at the 
school were a result of having a forward-think- 
ing curriculum coordinator who foresaw the 
changes and began putting them into place 
before the arrival of the MSPAP. The teachers 
were very well-informed about these changes 
and how they came about, as evidenced by 
their comments: 

[We] knew what the trends in education 
were going to be. We knew about Writer's 
Workshop and some of these things we have 
been implementing, not only for the past 
two years, but for many years. So, changes 
in the curriculum have not happened over- 
night. It has been something very gradual. 

The innovations to curriculum have been 
built on a solid foundation of theory and re- 
search. The changes appear to have been com- 
municated clearly to faculty members since 
curricular innovations were consistent among 
faculty members, all of whom were well- 
informed about current theory and practice. 



While the unique ways in which five 
schools made changes have been described, the 
discussion that follows provides converging 
evidence as to how the MSPAP affected the 
types of changes nnde, how these changes 
were made, and how affective behaviors 
changed as a result. 

Realignment of Curriculum, Materials, and 
Classroom Cultures to Match Features of 
the MSPAP 

The authors hoped that by interviewing a 
number of knowledgeable individuals within 
each school, a complete picture of the types of 
changes would emerge. For this reason, the 
comments made by teachers and administrators 
concerning instructional chan^ js were catego- 
rized as follows: nature of students' instruc- 
tional tasks, instructional methods used by 
teachers, texts used, learning environment in 
the classroom, assessment, organization and 
planning, and test preparation. 

All schools, and all individuals within each 
school, reported that the nature of instructional 
tasks designed for students and the instructional 
methods used by teachers have changed as a 
result of the MSPA*\ Similarly, there is evi- 
dence that the types of texts used in the schools 
and the learning environment in the classroom 
have changed as a result of the MSPAP. To 
find out how the MSPAP has affected instruc- 
tional change, the four types of changes thv'i 
provided converging evidence between and 
across all five schools will be discussed in 
more detail: (1) the nature of instructional tasks 
used in the classroom, (2) teachers' instruction- 
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al methods, (3) texts used, and (4) the learning 
environment. 

Nature of Instructional Tasks 

Although the teachers and administrators 
described many instructional tasks that were 
introduced as a result of the MSPAP, three 
facts about these tasks emerged repeatedly 
across all schools and interviews: (1) students 
had more opportunities to write; (2) there was 
more emphasis on personal response to read- 
ing, and (3) students had more choices in both 
reading and writing. Each type of task will be 
described more fully and evidence provided in 
terms of interview comments and supporting 
documents that link the changes to the MSPAP. 

Increasedwriting opportunities. Teachers in 
all schools indicated that writing is emphasized 
more since the inception of the MSPAP. After 
reading, students are frequently asked to re- 
spond to writing prompts, write in response or 
dialogue journals, or to doprewriting, drafting, 
revising, proofreading, and sometimes publish- 
ing. One third-grade teacher at School E noted: 

I would say die greatest change that I have 
made has been in the writing aspect. Such a 
focus has been put on writing, and writing 
from prompts, and writing connected to 
every aspect [of the curriculum], that practi- 
cally every breath they [the students] take 
they write about ... the kids are starting in 
kindergarten, by the time they get to third 
grade they are better writers than they have 
been in the past. 

The increased emphasis on writing may be 
directly linked to the Writing Outcomes as 



described in the Maryland School Performance 
Program (1991): 

The students will develop as writers through 
frequent writing experiences and many op- 
portunities to interact with each piece of 
writing, having had occasions to prewrue, 
draft, revise, and proofread, (p. 29) 

Another clear trend appeared to be that 
students were expected to organize their think- 
ing through the use of graphic organizers such 
as story maps and Venn diagrams. Teachers 
used the organizers in a variety of ways—some 
in the prewriting phase of the writing process, 
and others in helping students organize infor- 
mation when critically examining a text (i.e., 
comparing and contrasting characters). The 
reading and language arts portions of the 1991 
MSPAP required students to use graphic orga- 
nizers in similar ways (Maryland State Depart- 
ment of Education, 1991). 

Some schools indicated that they were 
involved with process writing prior to the 1991 
administration of the MSPAP. However, the 
teachers and administrators in these schools 
agreed ',hat the amount of writing has increased 
since '.he MSPAP and that the integration of 
writing across the curriculum was linked to the 
MSPAP. This change may not be a result of 
the 1991 MSPAP— which only included sub- 
tests on reading, writing, and mathe- 
matics—but it may be a result of the 1992 
MSPAP, which did include subtests on social 
studies and science that relied heavily on 
writing as a means of assessment (Field Obser- 
vation of MSPAP Test Administration, May, 
1992). The reading specialist at one school 
noted: 
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We're doing a lot more writing ... we did 
a lot with the writing process— Writer's 
Workshop— prior to MSPAP. Now we 
have added onto that, training teachers how 
to write prompts in response to literature 
assignments, prompts in science. We're 
trying to do that in all areas and integrate 
writing into all the content areas as well as 
reading. 

Emphasis on personal response to reading. 
The Reading Outcomes Model of the Maryland 
School Performance Program (1991) suggests 
that students should be able to read a variety of 
texts for a variety of purposes (italics added for 
emphasis): 

For example, students should be able to 
demonstrate that they can orient themselves 
to read for literary experiences (novels, 
short stories, plays, etc.), to be informed 
(content texts, articles, editorials, etc.), and 
to perform a task (following directions, 
etc.). Additionally, readers interact in dif- 
ferent ways with text. Students should be 
able to demonstrate their ability to interact 
through four reading stances: global under- 
standing (i.e., initial impression), develop- 
ing interpretation (i.e., more complete 
understanding by revisiting the text), per- 
sonal reflection/response (i.e., subjective 
consideration as it relates to personal knowl- 
edge), and critical stance (i.e., objective 
consideration of text) in order to construct, 
examine, and extend meaning, (pp. 19-20) 

Most schools and teachers in these schools said 
that an emphasis on personal response was an 
outgrowth of the MSPAP. Tasks that encour- 
age personal response included both oral and 



written responses. According to some third- 
grade teachers at School B: 

They [the students] have time to read every- 
day. Sometimes we'll do reading workshop 
where they'll have a period of 20 to 30 min- 
utes to just sit there and read, and then 
they'll do journal writing about that particu- 
lar book. 

These activities reflect the Reading Outcomes 
Model of the MSPP in that students are pro- 
vided with extended periods of time for reading 
(i.e., orienting themselves to read for the 
literary experience) and then are provided with 
time to respond to what they have read in their 
journals (i.e., the personal reflection/response 
stance). Additionally, students are responding 
to texts orally. 

Student choice in reading and writing. 
Teachers and administrators in most schools 
said that giving students choices of reading and 
writing material is a direct response to the 
MSPAP. One fifth-grade teacher at School B 
explained that after she introduces a number of 
theme-related books to her students through a 
book talk, she provides for student choice in 
reading: 

[If] I have four novels going in my class- 
room, the kids will make three choices. I 
always try to place kids with their first 
choice, but I do not like to have groups 
smaller than four, and I don't like to have 
groups larger than eight [reading the same 
novel]. So [the students] may get their sec- 
ond choice and read their first choice novel 
on their own. Those groups will meet at 
literature circle time to discuss their novel. 
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Other teachers described choice in reading and 
writing as options available to students in 
response to what they are reading. For exam- 
ple, students may choose to read another book 
by the same author or they may choose to write 
a letter to the author or write a poem or write 
a sequel to the novel as their response to what 
they have read. 

Although choice is not specifically outlined 
as a reading or writing outcome in the Mary- 
land School Performance Program, choice is 
provided for on the assessment itself (Maryland 
State Department of Education, 1991; Field 
Observations of MSPAP Administration, May, 
1992). Both third- and fifth-grade students are 
given choices of texts to read on different 
portions of the assessment, as well as a choice 
of type of written response. Thus, it seems that 
the inclusion of student choice in instructional 
activities is a direct outgrowth of the MSPAP. 

Instructional Methods 

Schools were originally selected as exem- 
plary schools that employed a specific type of 
instructional change, but in all schools three 
instructional methods were being implemented 
to some degree: (1) thematic, literature-based 
reading instruction that develops students' 
strategic awareness, (2) writer's workshop, and 
(3) integrated instruction. 

Literature-based instruction that develops 
strategic awareness. Teachers and adminis- 
trators in each school said their instruction was 
literature-based and included strategic instruc- 
tion as well. The fifth-grade teachers at School 
C described their instructional program as 
follows: 



Tl: We're using an integrated approach with 
thematic units. We work [in] the types of 
skills and strategies that are headed in the 
direction we need to go and that are covered 
by the MSPAP. 

T2: Working on strategies instead of discrete 
skills is a giant difference in the way things 
were prior to MSPAP. 

Teachers in most schools also said that includ- 
ing strategic instruction was a substantial 
change. Third-grade teachers at School D 
described the instructional changes they have 
made: they now focus not only on what they 
teach but how they teach it: 

Tl: Instructional changes I think we have are 
that the children [are] writing in response [to 
what they read] and dealing a lot with strate- 
gies—teaching them the BDA— what you do 
Before, During, and After [reading], and 
teaching them ways to use their reading 
skills when they are not in reading group. 

T2: I know that my teaching style has changed. 
I do think-alouds, and I model. I sit down 
and I will say , "I have to write a note here. " 
I will sit down and when it is time for silent 
reading . . . [previously] when they [the stu- 
dents] were all reading I was doing this and 
that because I had to get it done. Now I sit 
down and read. I will start laughing out 
loud, and I know I didn't do that before. So, 
I think that has changed my whole teaching 
style. It is modeling for them .... 

These instructional changes and methods are 
reflected in the Reading Outcomes Model of 
the Maryland School Performance Program 
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(1991) in that one of the reading outcomes is 
directly related to strategic behavior: 

Students will demonstrate their ability to 
construct, extend, and examine meaning for 
a variety of texts by using strategic behavior 
and integrating both their prior knowledge 
about reading and topic familiarity, (p. 21) 

Writer's workshop. As noted earlier, the 
Maryland School Performance Program's 
Writing Outcomes include provisions for the 
writing process: 

The students will develop as writers through 
frequent writing experiences and many op- 
portunities to interact with each piece of 
writing, having had occasions to pre write, 
draft, revise, and proofread, (p. 29) 

In light of the changes in students' instructional 
tasks and the mandate from the MSDE to 
incorporate the writing process into the rep- 
ertoire of instructional methods, it is not sur- 
prising that teachers and administrators said 
that writer's workshop was an instructional 
change that came about in response to the 
MSPAP. The third-grade teachers at School B 
described one way they integrate the writing 
process into their thematic units: 

There are usually one or two majcr writmg 
activities to go with each unit. In the mys- 
tery unit we do the major writing of a mys- 
tery story, and they [the students] start with 
planning and thinking— what they're going 
to write— and then they go through the 
whole writing process, doing a rough draft, 



peer editing, teacher editing, final copy, 
and making a cover. 

While not all items on the MSPAP involve the 
writing process directly, assessment items in 
all subject areas are followed by writing 
prompts. For example, after students have 
performed a science experiment, a problem- 
solving task in mathematics, or have completed 
reading a narrative, they are supplied with a 
writing prompt that tells them what form their 
written communication should take. They 
might be asked to write in the style of a busi- 
ness letter, a friendly letter, a diary entry, or a 
report. They are told the intended audience, 
the topic of the writing, and the purpose of the 
communication. These prompts are often long 
and complicated. So, the teachers at School B 
not only let their students take some of their 
writing completely through the writing process 
but have included instructional methods that 
help students understand writing prompts. 
While such direct instruction is not provided 
for students, all the teachers said their reper- 
toire of instructional methods included instruc- 
tion in the writing process. 

Integrated itistraction. Teachers' instruc- 
tional methods were also affected by the 
MSPAP in that nearly all schools had begun to 
integrate their instruction across the entire 
curriculum. One supervisor of instruction even 
explained that their county was moving toward 
integrated instruction even at the high school 
level. The link between integrated instruction 
and the MSPAP is not stated in the Reading or 
Writing Outcomes. It may, however, be linked 
to an outcome that addresses the need for 
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students to read not only for the literary experi- 
ence but for other purposes as well: 

Students will demonstrate their ability to 
vary their orientation to the text by interact- 
ing with a variety of texts for different pur- 
poses. Students will read for literary exper- 
iences (novels, plays, short stories), to be 
informed (content texts, articles, editorials), 
and to perform a task (follow directions, 
some action required of students), (p. 21) 

On the MSPAP, this reading outcome may 
occur during a science experiment or a math- 
ematics problem-solving situation. Thus, the 
reading outcome and the nature of the assess- 
ment itself may have combined to produce the 
changes in instructional methods. A fifth-grade 
teacher at School B describes the way their 
team has integrated their instruction: 

In the first quarter, everything was instruct- 
ed around the environment, "Our World." 
The kids read nonfiction as weil as fiction 
about the environment, and I placed my 
students [into groups] so that they were 
looking at a particular issue. Whether it be 
acid rain, whether it be rain forest, whether 
it be recycling, reusing, we did science 
experiments in language arts; we collected 
data in language arts; we collected data in 
math. Everything tied together. The best 
part was hearing one of my students say, "I 
don't know if I'm in math or language 
arts." That was the best comment. 

This method of instruction is quite different 
from using thematic units. Thematic units 
imply that all of the literature-based assign- 
ments are related to the same topic, but the 



theme may not carry over into other subject 
areas. By using integrated teaching methods 
and thematic, literature-based instruction, these 
teachers base their entire curriculum on a 
common theme. 

Texts Used in the Classroom 

The third type of instructional change was 
the emphasis on using authentic literature 
rather than the more traditional basal readers. 
These basal reader series structured the entire 
curriculum for teaching reading and provided 
all of the necessary materials for implementing 
the program (Otto, Wolf, & Eldridge, 1984). 
Not suiprisingly, the design of a basal series 
has been found to influence the type of in- 
struction that teachers provide for students 
(Barr & Sadow, 1989). Thus, in those schools 
using basals, students were taught to read by 
having a controlled number of words intro- 
duced to them at a time. Because it is difficult 
to find authentic pieces of literature that use a 
limited number of new words, stories in basal 
readers were often contrived. Researchers 
theorized that the simplistic language patterns 
and limited vocabulary of these readers might 
have adverse effects on children's language 
development (Chall & Squire, 1991). In re- 
sponse, publishers began including authentic 
literature within their basal reader programs, 
but often would alter the original text to com- 
ply with readability formulas and space limka- 
tioas (Chall & Squire, 1991). 

More recently, publishing companies have 
developed "literature anthologies" that are 
made up of authentic literature and include the 
entire text, along with reproductions of the 
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original illustrations (Kucan, 1994). This type 
of basal reader was used by most of the teach- 
ers (other than the teacher of the gifted and 
talented) at School A. The other schools used 
authentic literature— also called "trade books" 
or "novels." 

The teachers and administrators we spoke 
with were well-informed about the rationale 
behind the use of authentic literature. The 
third-grade teachers at School D noted the 
problems they saw with the more traditional 
basal readers after they had made the switch to 
authentic literature: 

Tl: After we'd gotten into the trade books, we 
started to see what was missing from the 
basals. 

T2: ... the quality of the literature . . . 

T3: The kids were much more enthusiastic. 

Tl: Plus, we found that they [the publishers of 
basal readers] would water down stories in 
the basal. They would take an excerpt from 
a story and put it in the basal and when you 
compared it to the real story they would 
water it down and leave out the good lan- 
guage. ■ 

The curriculum coordinator at School E ex- 
plained further: 

Whole texts are being used instead of short 
little excerpts to teach reading. I think grade 
schools don't look as much to exposition. 
The MSPAP probably has moved us more 
to make sure that we're focusing on that 
too. 



She notes that not only has the MSPAP influ- 
enced instruction by encouraging that longer, 
authentic texts be used, but that texts used 
during instruction are no longer exclusively 
narrative. Administrators and teachers at all 
schools noted this change. 

The role of the MSPAP in evoking change 
may be related to the fact that the assessment 
itself does not use the typical short isolated 
paragraphs that traditional standardized tests 
use as texts in pssessing reading ability. In- 
stead, the MSPAP offers lengthier, authentic 
texts bound together into a booklet known as 
the "Student Reading Book" (Maryland State 
Department of Education, 1991). In accor- 
dance with the Reading Outcomes of the Mary- 
land School Performance Program (1991), a 
variety of texts are used on the MSPAP (e.g., 
plays, short stories, content articles, editorials, 
recipes) as well. 

Changes in Learning Environment 

Another substantial instructional change has 
been the way in which students, teachers, and 
parents are involved in classroom learning. 
Although the quality of the learning environ- 
ment depends on the instructional changes 
already discussed, the term "learning envi- 
ronment" implies the totality of the learning 
situation as opposed to its component parts. 
That is, the tasks performed by students and 
the methods used by teachers both contribute to 
the academic conditions that create a learning 
environment, which can be viewed as an ag- 
gregate of social, cultural, and academic condi- 
tions that influence the learning experience for 
a given individual. The effects of the MSPAP, 
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then, extend beyond student learning; they 
affect teachers and parents as well. 

Students. Across all schools, .eachers and 
administrators reported that the learning envi- 
ronment had changed because students were 
given more voice in how lessons evolved. 
Students also had been given more choice in 
classroom learning. The supervisor of instruc- 
tion at School D described the effect of this 
change on students: 

I think the whole atmosphere of a class- 
room—the learning climate— is important. 
' Since you are doing so many more activities 
that are more student-facilitated than teach- 
er-directed, then you are much more likely 
to have a classroom atmosphere that might 
appear to a casual observer to be noisier and 
less focused, but in reality on closer inspec- 
tion you are seeing lots and lots of different 
learning all at once. For many children, 
having the opportunity for the first time to 
make choices, and have the freedom to 
make choices in what they do— within 
certain bounds and certain standards— will 
be accepted. 

The supervisor's perceptions were verified by 
the fifth-grade teachers at School D. (Note that 
these are the same teachers who were frustrated 
by the sudden changes they were expected to 
make.) Here, they note the positive effects that 
changes in the learning environment produced: 

Tl: In the past, in other schools a supervisor or 
principal would say, "We expect you to have 
the other kids working on assignments or 
dittoes or a workbook when you're working 
with a reading group." They [the students] 



had to be silent, or quiet, and do exactly 
what you told them to do. The kids now 
have more choice. There are structured 
things they have to do, but they are starting 
to see that you can choose to read, and you 
can choose to write, and you can choose to 
do it without being directed to do it. 

T2: I am seeing the possibility of more choices 
now. Whereas in the past it was, "my way 
or the highway. " If I want a bunch of kids to 
read a chapter from their current book, they 
can decide whether or not they want to read 
it on their own, or read it with two other 
people, or read it out loud, or read it silent- 

iy. 

The other change that all schools noted was 
the increase in student interaction and involve- 
ment in the lessons. The fifth-grade teachers at 
School E highlighted the fact that if students' 
roles in the learning environment were to 
change, the teacher had to permit the social 
rules that govern classroom instruction to be 
altered (italics added for emphasis): 

I think the one important thing that I found 
to be very helpful in my classes is to allow 
the students to have the opportunity to have 
input into how we should do a lesson- 
student involvement. 

Comments by a third-grade teacher at School 
D also show that student interaction in the 
learning environment is a change from the 
past: 

Children [are] helping other children. It is 
really funny because I remember in the old 
days when you would say talking wasn't 
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allowed, and the children were not allowed 
to help each other— and now we see that life 
isn't like that. You have to know how to 
work with people and ask for help . . . Just 
in the last few years it has been so much 
better because they are not sitting there with 
"fences" around their work— people don't 
live like that. 

Although nothing in the Reading Outcomes 
of the MSPP accounts for the increases in 
student involvement, the MSPAP assessment 
itself may have induced such change. The 
MSPAP provides opportunities for students to 
work cooperatively in groups during portions 
of the test and also allows for student choice 
within the assessment (Maryland State De- 
partment of Education, 1991; Field observa- 
tion of 1992 MSPAP administration, May, 
1992). Based on these comments and the for- 
mat of the MSPAP, it seems that not only has 
the assessment altered instruction, it has 
changed the way students behave in their 
learning environment as well. 

Teachers and administrators. Just as stu- 
dents' roles appear to have been affected by the 
MSPAP, so too have the roles for teachers and 
administrators. Not surprisingly, if students are 
being provided with greater opportunity for 
decision making and choice, then teachers must 
be relinquishing some of that power. The 
teachers and administrators in all five schools 
support the notion that teachers are shedding 
some of their more traditional roles and func- 
tioning more as coaches and facilitators than as 
authority figures. The curriculum coordinator 
at School E noted that during literary discus- 
sions, teachers are relinquishing the more 



traditional role of inquisitor so that student 
interaction can occur: 

[The MSPAP has] geared our teachers to let 
kids respond in response groups so that 
there is more interplay among kids, and 
teachers [are] taking a "stand back" role 
and letting kids get into the discussion. 

Administrators have also changed their 
roles in creating learning environments; they 
have had to be supportive and allow teachers to 
become risk takers. The comments of the 
principal at School C substantiate this role: 

I go to the business of being the supportive 
force— allowing for creativity and risk 
taking. I know resources are important, and 
you want them there, but it [change] can be 
done without all those resources too. It has 
been done, but I think the more important 
thing is the support for school [staff]— to let 
them identify their needs and to see how 
they feel they can meet those needs and give 
them a chance, to be accountable, but to 
give them a chance to see if they can devel- 
op their plan. 

The comments of all the admirstrators re- 
flected the basic tenets of the Maryland School 
Performance Program's (MSPP) outcome- 
based approach to school improvement. This 
document is available to administrators in the 
Maryland School Performance Program Guide 
(1991). The Guide describes implementation 
procedures at the school level and was devel- 
oped to help administrators implement the 
suggestions for change made by "The Report 
of the Governor's Commission on School 
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Performance" (Sondheim et al., 1989). The 
introduction to the Guide mirrors adminis- 
trators' comments: 

School improvement in Maryland is driven 
by a practical school-based instructional 
decision making process. Each school is the 
center of instructional decision making for 
its students. School-based decision making 
is supported by a RAND Corporation study 
of 40 years of school reforms {Steady Work: 
Policy, Practice, and Reform of American 
Education, 1988) which concluded that: 

1. Reforms must take place at the school 
building level. 

2. Reform efforts must be given enough 
time to succeed or fail. 

3. School staffs must be involved in the 
design of reform. 

4. We must not expect quick and easy solu- 
tions. Significant and lasting change will 
take time. 

5. School solutions may make schools] pro- 
grams very different. 

Each school receives a new professional 
role with demanding responsibilities. This 
new role calls for the development of a 
common vision, data-based decision mak- 
ing, risk taking, creativity, change, and the 
selection of proven teaching strategies and 
programs so that each student may succeed, 
(pp. 1-2, 1-3) 

Recommendation #5 of the Report links di- 
rectly to the MSPP Guide (1991) as well as to 
the comments made by the administrators: 



We recommend, as an important step to- 
ward school improvement, the elimination 
of rules, regulations and other strictures that 
constrain school staffs in applying their 
professional abilities and creativity to the 
task of teaching children (p. 4). 

All the administrators in this investigation 
have taken seriously their responsibility for 
school improvement that these state policies 
sanction and are ensuring that such changes 
occur in their schools. 

Parents. Many parents and teachers said 
that an increased emphasis was being placed on 
involving parents in the learning environment 
both at school and at home. Third-grade teach- 
ers at School E enlrted the support of parents 
in order to implement a read-at-home program 
in which students were encouraged to read for 
at least 15 minutes every night. Parents were 
asked to document students' efforts and to 
interact with their children about what they 
read. Third-grade teachers at School C asked 
parents to help in the classroom by assisting 
small groups with library' work, art projects, 
and reading projects. 

The reliance on parental support is not a 
new idea, but in these schools it may be linked 
to the MSPAP because some parents were con- 
cerned about the changes being made in the 
schools. Parents were raising questions because 
they were unfamiliar with the tasks students are 
being asked to do, the methods being used in 
the classroom, the longer texts their children 
were being asked to read, and the new roles 
their children had assumed in the learning envi- 
ronment. The principal at School C explained 
that the cohesive learning environment she was 
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trying to create was contingent upon the under- 
standing and support of all involved: 

I guess I always feel that you have the "vi- 
sion," but it cannot be carried out if you 
don't have the support of the staff and the 
community and everybody who's a part of 
the school. That trust has to be developed. 
That whole program has to be developed, 
and I think that is [accomplished] by giving 
them the responsibility and input so it's a 
case of the staff, it's also a case of the kid, 
it's also a case of the parents, and it's the 
community— it has to be the whole team 
effort— otherwise it can't be done. How did 
we go about organizing that? . . . You have 
to develop that special environment among 
your staff, and you have to have that com- 
mon mission and [know] where you want to 
head, and that's what we worked on first. 
Then gradually bring in the parent. They 
have to buy into it [the common mission]. 
They have to be supportive. Again, every- 
thing won't be perfect; but again, the will- 
ingness for everybody to talk and to be cre- 
ative and to be willing to try is crucial. Get 
parents in here. Let them see what's going 
on, and let them be involved with it. 

In addition to permitting parents to be a part 
of the learning environment itself, workshops 
were developed in some schools to educate 
parents about the new methods of instruction 
and the rationale behind their use. Several 
schools sought "Parent Involvement" grant 
funds to help them educate parents about the 
curricular changes; School A and School B 
reported that they had received such grants. 

School E organized a similar program of 
education for parents. However, the principal 



noted that because many parents are immi- 
grants working at blue-collar occupations 
requiring shift work, parent in-services had to 
be staggered throughout the day to accommo- 
date work schedules. Identical sessions were 
provided in the mornings, afternoons, and 
evenings so that parents could attend sessions 
when it was convenient for them. 

Including parents in the learning environ- 
ment and educating them about the changes in 
the schools cannot be directly linked to the 
MSPAP assessment or MSPP documents. 
However, administrators and teachers have 
apparently realized that if instructional changes 
are to be understood and accepted by parents, 
provisions must be made to include parents in 
and inform them about those changes. 

Professional Support for Teachers at the 
District/County Level and School Level 

The instructional changes described above 
were costly, both in time and money. From the 
outset, the state of Maryland placed the largest 
burden for facilitating change on local school 
systems, as was evident in the Report (Sond- 
heim et al., 1989). Since 23 of Maryland's 24 
school systems are organized by county, the 
term "local school system" includes the county 
and school levels. 

Thus, the local school systems were respon- 
sible for providing the professional develop- 
ment programs their teachers needed in order 
to make the instructional changes requested. 
The Report justified the inevitable inequities 
among the school systems as a means of identi- 
fying those systems in need of greater support. 
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Table 4. Sources of Ideas for Change Between Schools and Proportion of Faculty Reporting Sources 
Within Schools. 



Schools 



Source of 
Change 


School A 
n = 2 


School B 
n = 4 


School C 
n = 3 


School D 
n = 5 


School E 
n = 6 


TOTAL 


Professional 














Materials 














State 


1.00 


0.00 


0.00 


0.00 


0.17 


0.23 


County 


0.50 


0.50 


1.00 


0.80 


0.33 


0.63 . 


School 


0.50 


0.75 


0.67 


1.00 


0.17 


0.62 


Professional 














Development 














Activity 














State 


1.00 . 


0.50 


0.33 


0.00 


0.50 


0.47 


County 


0.00 


0.50 


0.67 


0.80 


0.67 


0.53 


School 


0.00 


0.50 


1.00 


0.40 


0.50 


0.48 


Discuss/Share 














County 


0.00 


0.50 


0.33 


0.80 


0.00 


0.33 


School 


1.00 


1.00 


1.00 


0.80 


0.83 


0.93 


Original Ideas 


0.00 


0.75 


0.33 


0.60 


0.50 


0.44 



Much of the burden for implementing 
instructional change did lie at the local school 
system level. Four broad categories of sources 
of ideas for teachers and administrators existed: 
professional materials, professional develop- 
ment activities, informal discuss- a and shar- 
ing, and original ideas. Table 4 shows the 
proportion of teachers* and administrators' 
responses within each school that identified 
similar sources of ideas for change at the state, 
county, and school level. 

In addition to providing information about 
where ideas for change were derived, teachers 



and administrators were asked to describe how 
these changes were financed. Table 5 shows 
where the resources for making these instruc- 
tional changes came from as well as the pro- 
portion of responses within each school that 
corroborated these statements. A more detailed 
discussion of resources follows. 

Professional Materials 

Each of the five schools indicated that 
professional materials (i.e. , professional books, 
curriculum guides, videotapes of instructional 
methods) helped them develop ideas for mak- 
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Table 5, Resource Availability and Proportion of Faculty Within Schools Reporting Sources of Resources 



Schools 

School A School B School C School D School E 



Resource 


n = 3 


n - 5 


n = 3 


n = 5 


n = 6 


TOTAL 


Professional 

a rial 














State 

County 

School 


0.33 
0.67 
0.67 


0.00 
0.20 
1.00 


0.33 
1.00 
0.33 


O.OO 
0,40 
1,00 


0.33 
0.67 
0.83 


0.20 
0.59 
0.77 


Professional 
Development 
Activity 














State 

County 

School 


0.33 
1.00 
0.67 


0.20 
0.60 
0.60 


0.00 
0.67 
1.00 


0,00 
0.80 
0.40 


0.00 
1.00 
0.83 


0.11 
0,81 
0.70 


Grants 














Federal 


Chapter 1 


none 
reported 


none 
reported 


none 
reported 


Head-Start/ 
Chapter 1 




County 


Parent Involve- 
ment grant 


Parent Partnership 
Grant/Lit. Based 
Curriculum Grant 


Art teacher 
obtained 
grant to 
integrate 
curriculum 


Mini-grant tied 
to MSPAP 


Success for 
Every 
Student 
grant 




School 


none 
repo.ted 


none 
reported 


none 
reported 


PTA support/ 
discretionary 


none 





funds reported 



ing instructional changes. In addition, counties 
bought many copies of trade books for class- 
rooms; all materials were primarily paid for 
with county- and school-level funds. Counties 
were quite helpful in providing professional 
materials. The reading specialist at School D, 
for example, said that many of their ideas for 
change come from a language arts guide: 



Most of the ideas [for changes] come from 
the county level where we have been given 
the guidance from the committee working 
on, or developing an Integrated Language 
Arts Guide. This [guide] is direct* / in line 
with MSPAP. 

Additionally, professional books were 
available at either the county or the school 
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level. Teachers and administrators at several 
schools said they could order from a profes- 
sional library in the county. The curriculum 
coordinator at School E said the principal had 
ordered many professional books for each 
member of the faculty: 

We bought all of the Heinemann books. I've 
got Invitations, I've got Lasting Impres- 
sions, we've got In the Middle—and the 
teachers have them too. We've got all the 
stuff on graphic organizers [that the princi- 
pal] ordered from the state and paid for 
herself, and had them all laminated so every 
teacher has their own set of those. Anything 
that I put on the list that we want, we get 
multiple copies of . . . Lessons from a 
Child, all of the stuff on the writing process 



Thus, it seems that access to professional 
materials was needed— and it was provided— in 
order for teachers and administrators to make 
mandated instructional changes. 

Professional Development Activities 

A great deal of support was also needed in 
the form of professional development activities, 
including teacher in-services, workshops, and 
model lessons. These activities were provided 
primarily at the local level, but the state sup- 
ported some through the Maryland State De- 
partment of Education. Two fifth-grade teach- 
ers at School B said they developed many ideas 
as a result of their participation in the Govern- 
or's Academy of Mathematics and Sciences one 
summer. The Governor's Academy was begun 
in the summer of 1989 as a means of educating 



exemplar teachers of mathematics, science, 
and technology across the state. It was funded 
as a start-up project that was operated by the 
Maryland State Department of Education in 
which teachers learned about instructional 
innovations as well as current content area 
research and theory. The teachers of School B 
said their experiences helped them see more 
clearly how integration might occur: 

Last summer [another fifth-grade teacher] 
and I attended the Governor's Academy of 
Mathematics and Sciences, and we began to 
see a clear picture from that— being im- 
mersed in math and science and writing 
outcomes and the different themes and the 
different ideas . . . 

Other activities provided by the state were 
available at School A in the form of univer- 
sity-provided programs. Although not funded 
directly by the state, university-based programs 
provided by the University of Maryland were 
categorized as professional development activi- 
ties provided by the state because the universi- 
ty is a state agency. 

Much of the expertise for providing ideas 
for change appeared to be available at the 
county level for each school. The principal at 
School D explained that their county had devel- 
oped MSPAP curriculum specialists who were 
available to work with teachers in the county: 

We spent a lot of time with our MSPAP 
teachers here in the county. We did model- 
ing lessons for teachers. We released teach- 
ers here for direct instruction with those 
folks so they could simply talk to them and 
learn. We had county-wide inservice meet- 
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ings as well. We have sent our folks to as 
many workshop meetings as we could af- 
ford to have them go to. 

Finding time to instruct teachers seemed to 
be particularly difficult; however, these admin- 
istrators shared some of the innovative ways 
they managed to do it. At School B, the princi- 
pal described how she got teachers released: 

I came up with really interesting ways to 
release teachers and get them out of the 
building because I really do feel that al- 
though they shouldn't be out of the class- 
room, they need to be inserviced. First of 
all I use teachers in the building with exper- 
tise to mentor the ones that are available . . 
. . The assistant principal and I go in and 
we teach classes so that these teachers can 
be released. There are some specialists who 
don't have class at [a given] time— we'll 
say, "Will you help out?" 

At School C and others, they also had release 
days from the central office for working on 
professional development. Thus, the adminis- 
tration at the county and school levels did 
provide the necessary staff development to 
keep faculty updated on theories and methods 
for making changes. 

Informal Discussion and Sharing of Ideas 

Teachers also got ideas from informal 
discussions and sharing with others; the oppor- 
tunity to share and discuss ideas was most 
available at the school level. The difficulty was 
finding time to do it. The principal at School E 



explained the unique "block planning sessions" 
that occurred in her school: 

The teachers have 3 hours and 45 minutes 
off during the student day in a student week 
[for planning]. So sometime between Mon- 
day and Friday all of them have block 
planning, and that's not an unusual thing in 
elementary schools, but I've used the cur- 
riculum coordinator [within the school] who 
has a doctorate in reading to meet with them 
for 45 minutes every other week and plan. 
Her [the curriculum coordinator] job is to 
sit and act as a curriculum specialist in the 
field of reading. She'll bring in reading; 
she'll bring in writing, oral language skills. 
She's full of suggestions. She'll maybe 
[share] materials that have come in, or [pro- 
vide] suggestions on how to use graphic 
organizers, or she may introduce a new 
graphic organizer. She acts as a catalyst and 
teachers will sit and talk about the things 
they've done. 

This opportunity for constructive dialogue and 
planning is certainly ideal. Each of the other 
schools seemed to have an equivalent support 
system in place. At School D for instance, the 
reading specialist noted that they have an 
Integrated Language Arts Support Group. 
Additionally, some schools gave teachers 
chances to discuss ideas with county supervi- 
sors. The fifth-grade teachers and the reading 
specialist at School B said their talk with super- 
visors enabled them to seek information about 
new ideas and to see if others could be tried. 
They were excited about the support they had 
received when they wanted to try to completely 
integrate their curriculum: 
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Tl : I remember sitting there [at lunch] talking to 
the math supervisor and we said, "But you 
can't get everything in. We're going to have 
to change some minutes along the line.'* 
[Our science supervisor] said, "That's no 
problem. " And I think that was like a signal 
to us . . . 

T2: ... that we had full support. 

Tl: Right, it was like they said "Go ahead and 
try it! " It's not like OK we've got this county 
curriculum and that kind of thing. [Like] we 
have to teach "X" amount of minutes for this 
. . . and we're saying, "Look, we want to 
try something, but we're going to have to 
change the way the county's thinking." 

T2: ... All three of them [county supervisors] . 
. . it was like a blank slate they gave us. 

Thus, it seems that not only was the opportu- 
nity to share ideas with other teachers impor- 
tant, but the support of the county supervisors 
was important for the discussion and sharing of 
ideas as well. 

Original Ideas 

Many of the teachers and administrators 
said they had to develop many of their instruc- 
tional ideas on their own, both at the level of a 
single lesson or for an entire curriculum. For 
example, the fifth-grade teachers at School B 
who wanted to totally integrate their curriculum 
described how they came up with their ideas: 

Tl: We cut and pasted it [the curriculum] to 
meet what we thought would be our needs. 



T2: We got the OK to change the scope and 
sequence and to move different units around. 

The principal at School B explained that be- 
cause she encourages creativity and risk taking, 
teachers are willing to try new ideas. She also 
noted that many of her own ideas come from 
assessing the needs of her students: 

I just make them [ideas] up. I was a reading 
specialist for nine years. Where do I get 
ideas? Well, I guess I get them from needs 
assessment. We go around and you look at 
your school, and you think. Til pick out 
two or three areas to prioritize, and then I 
come up with ideas and they [the teachers] 
all buy in. The staff pretty well buys into 
most of them because they'll come up with 
them too. I think if you encourage creativity 
and risk-taking, people will come in here 
and say, "Hey, can I try this?" Like [one 
fifth-grade teacher] came and said, "Can I 
teach Science and Social Studies all day if I 
integrate all my other subjects [reading, 
language arts]?" And I said, "Yeah, that's 
great!" So we threw the schedule out. We 
don't teach by minutes any more, by block 
of time— can't do that. So, how do we come 
up with ideas? Why we just sort of brain- 
storm, and think about them, and read— I 
read articles all the time. I listen to the 
needs of the kids in the school. 

Thus, it seems that the county and school 
levels have played a major role in financing 
instructional changes. They have funded pro- 
fessional materials, professional development 
activities, and have provided opportunities for 
teachers to discuss and share their ideas. Build- 
ing administrators have played an important 
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role in securing additional funds to purchase 
materials through grants (see Table 5). Like- 
wise, the administrators played a key role in 
organizing and planning so that teachers could 
either plan together during the school day, as in 
the block planning sessions at School E, or 
outside the school day. Through a concerted 
effort between county administrators, school 
administrators, and teachers, these schools 
combined their resources to share ideas in 
order to make their new curriculum work. 

Affective Changes in Students' and 
Teachers' Behavior 

The third category of change in these 
schools dealt with the impact of the instruction- 
al changes on students and teachers. Three 
types of effects became apparent: effects on 
emotional state, motivation/attitude, and stu- 
dent classroom behaviors. 

Emotional State 

Teachers in several schools described the 
emotional effects on students primarily in terms 
of how they would be affected by the test after 
the instructional changes were put in place. 
Several thought that students would feel more 
comfortable with the test since many of the 
items on it would reflect typical daily instruc- 
tion. 

The primary emotional effect, however, 
was on the teachers. Most admitted that the 
test, and the instructional changes required as 
a result of it meant frustration for them. Many 
teachers noted that they were nervous and even 



frightened by the test. One individual at School 
E said: 

We had so many training sessions [in prepa- 
ration for the MSPAP] and one outcome, 
and I am not sure that it was very positive, 
was the anxiety that was expressed by the 
teachers in not knowing exactly how the 
results would be reported, and I think that 
they felt a degree of guilt or worry in what 
this would mean for them as teachers if the 
results were negative for School E. 

Perhaps the fact that these were state-mandated 
changes that were being implemented quickly 
brought about such anxiety. However, while 
many of the teachers expressed their frustra- 
tion, they felt that the changes were necessary 
nonetheless: 

Tl: They are good changes, and we are heading 
in the right direction there is no doubt about 
it. It is just frustrating. 

T2: Our problem is that we want to get there, 
and we know we can get there, and we want 
to get there now. 

Student Motivation and Attitude 

All schools, except School A, reported 
positive effects on students' motivation and 
attitude toward reading and writing. These 
views were consistent across schools. Howev- 
er, it must be noted that these changes in 
motivation and attitude are based on the reports 
of teachers, not students. The fifth-grade tea- 
chers at School D seemed to attribute this in- 
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creased motivation to the use of authentic 
literature: 

Tl: One thing with the novel approach [litera- 
ture-based instruction], the superiority of 
that is the children get to see a character 
develop throughout the whole story. By the 
end of a book, my children are so enthused 
about finishing it. . 

T2: I can say that within the 20 years, not count- 
ing these [past] three years, I have been 
doing literature, I don't think I ever had kids 
who really liked to read in reading class 
because the books stunk. 

T3: Now they can't wait to read out of those 
books! 

Tl : They love to read now. They were reluctant 
once. 

Teachers at several schools also reported a 
change in students' attitudes toward reading, 
but seemed to attribute the change, in part, to 
the different learning environment that was 
established. Likewise, they attributed part of 
the changes to such things as providing stu- 
dents with choice. The comments of the read- 
ing specialist at School D support this percep- 
tion: 

I see a difference when [the students are 
able to] choose. It is in the attitude, and that 
is one of the areas that MSPAP addresses; 
the attitude toward the language arts is more 
positive, because it is something they 
choose to do at times rather than always 
being dictated to during this time. 



As was noted earlier, one of the Reading 
Outcomes of the Maryland School Performance 
Program (1991) was that, "Students will dem- 
onstrate positive attitudes towards reading a 
variety of texts" (p. 21). Based upon the evi- 
dence above, teachers, apparently think this 
has happened. 

Many of the schools also indicated that 
positive attitudes were displayed toward writ- 
ing as well. The third-grade teachers at School 
D described how students' motivation for 
writing has been positively influenced: students 
are more confident in their writing as their 
motivation has increased: 

Tl: I think that [the students] are feeling more 
self-confident. They are feeling like they are 
more successful. 

T2: They #ant to read, and they want to write. 
They come in [and say], "Look what I did!" 
At recess they say, "Can I stay in the room 
and write?" 

T3: Yes, they say, "Can I stay in the room and 
write?" 

T2: I don't remember that they said things like 
that in years past. 

The third-grade teachers at School C expressed 
similar delight at their students' increased 
motivation for writing: 

Tl: I know there's been a positive impact with 
our program when we do our journals. In 
the past, if you gave them a journal starter 
they completed it with a sentence. Now they 
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go pages and pages and pages and pages and 
they just keep going and going. And they 
don't have to— it's not getting a 
grade— they ' re never graded , they ' re 
checked. So, they must be doing it from 
their own self-motivation. You'd think 
they'd write one sentence, put the pencil 
down and be done, but they go on and on 
and they illustrate it. 

T2: They [the students] love the novels. Writing 
is so natural. There aren't any more groans 
or, tt Oh, here we go again, a story ..." It's 
just like part of the routine. You brush your 
teeth every day, you eat lunch every day, 
you write every day. They don't know any 
different. This is life; you write every day. 

Student Behavior 

Teachers also reported changes in students' 
academic behavior, although there was little 
continuity across schools in terms of types of 
changed behaviors. The teachers and principal 
at School C noted that students' thinking skills 
had improved: 

Their thinking skills are better as is their 
creativity . You' re pulling things out of some 
students that you wouldn't get any other 
way. This way they all have to have some 
kind of input, and you learn that it's not 
always your smartest students that have the 
best answers. 

This type of observation is in line with rec- 
ommendations from "The Report of the Gov- 
ernor's Commission on School Performance" 
(Sondheimet al., 1989): 



Some of the data available in schools would 
be collected, analyzed and used by every 
superintendent and local board to decide 
what actions are necessary for each child to 
learn and make progress. This is particu- 
larly crucial at the primary and elementary 
levels, but it is important at every level of 
schooling (p. 13). 

Based on this statement, some counties have 
adopted "Success for Every Student" as the 
motto of their educational system. The third- 
grade teachers from School C felt that every 
child should be given the same opportunity to 
experience a solid program of instruction in 
which they can display what they know. 

The third-grade teachers at School B identi- 
fied specific literary behaviors as improved 
among their students: 

I think they're learning about literature. 
They're learning how to look at it from all 
different perspectives, and they're learning 
that they can find, if they're interested in a 
particular book, or interested in an author, 
there are places to find more to read. 

These comments imply that through the in- 
creased use of authentic literature and exposure 
to literary criticism at an elementary level, 
students are becoming more adept at reading 
from a literary perspective. 

The fifth-grade teachers at School D felt 
ritat their students had become more indepen- 
dent and self-directed in their learning because 
of the curricular changes: 

Tl . I think they are becoming more self-directed 
learners because of this. 
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T2: I think it leads them to be more independent 
workers and be better problem solvers with 
other members of their group, and indepen- 
dently on their own. 

Teachers in several schools noted that 
because the learning environment had changed, 
classroom conduct had improved and behavior 
management problems had subsided. The 
third-grade teachers at School C noticed a 
substantial change in their classrooms: 

Tl: There is a lot of movement, activity, you 
have to close your door. The noise level is 
so high, but if you're walking around and 
listening to the conversations it's all related 
to the project at hand. They're really into it, 
and they're all discussing it, and that's fine, 
and we like that. 

T2: We eliminate a lot of behavior problems be- 
cause they're talking. It's not, "Shhh, no talk- 
ing allowed." They're communicating and 
when it's time to get busy and work they've 
already got it all our of their system, and they 
get to work. The whole behavior management 
system has changed for the better. 

The fifth-grade teachers at School D cor- 
roborated these observations, attributing the 
decrease in behavior problems to the fact that 
students had greater ownership of the instruc- 
tional activities that they were participating in: 

1 have found that when working with anoth- 
er reading group, and they are doing inde- 
pendent seatwork, since they have greater 
ownership in a lot of ways with what they 
are doing, and more investment in it, I don't 
have the discipline problems that I used to. 



When it was just giving them dittoes and 
saying, "Here do these" they weren't as in- 
trigued. They weren't as interested because 
they weren't relevant. Consequently, I'd have 
to keep pulling students out of the group 
and disciplining more. Now my children are 
more self-directed. There are a few who 
will lose their attention span and maybe talk 
inappropriately, but it is so much less than 
it used to be. There might be more of a 
"buzz" in the room, but it's OK because it's 
productive. 

Overall, it seems that although the MSPAP 
may have made teachers feel frustrated and 
anxious, the instructional changes brought 
about by the MSPAP seems to have had very 
positive effects on students' motivation and 
attitude toward reading and writing. Instruc- 
tional changes may have also produced positive 
effects on students' academic behavior as well 
as their classroom conduct. 

CONCLUSIONS 

In this study, evidence showed that the 
Maryland School Performance Assessment 
Program has produced consistently positive 
effects on classroom practice. While there are 
substantial data to support this claim, we can- 
not claim wide general izability of these find- 
ings because our informants worked in elemen- 
tary schools that had been deliberately selected 
as exemplars of instructional change. Thus, 
conclusions should be considered relevant to 
those school communities. 

The evidence suggests that the types of 
instructional tasks, methods, materials, and 
learning environments were altered by teachers 
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and administrators. These alterations seem to 
reflect the nature of the MSPAP assessment 
itself and the state-mandated outcomes for 
literacy. However, it may also be that the 
assessment was designed to conform to instruc- 
tional practice. In any case, the comments by 
school personnel suggest that instructional 
alterations did occur in these schools. More 
specifically, instructional tasks were altered to 
give more writing opportunities, to emphasize 
personal response to reading, and to include 
student choice in such literacy tasks. Addition- 
ally, the instructional methods employed by 
teachers included thematic, literature-based 
instruction, writer's workshop, and integrated 
instruction. The use of authentic texts was also 
clearly present in these schools. Finally, the 
classroom learning environments reflected more 
student input and interaction during lessons and a 
less dominant role for the teacher. 

In order to produce such change in their 
classrooms, teachers had to participate in the 
design of the curriculum. Likewise, the teach- 
ers had to be willing to abandon incompatible 
methods of instruction and ideologies for new 
ones. The potential for tension between faculty 
and administration in such a situation is obvi- 
ous. Thus, instructional change also required 
major efforts in terms of administrative support 
at the district/county level and the school level. 
Substantial support in these schools was pro- 
vided through the purchase of professional 
materials that would describe the theoretical 
basis for the desired changes as well as practi- 
cal methods for doing so. Additionally, profes- 
sional development activities such as in-seivic- 
es, demonstration lessons, and workshops were 
essential for change. Teachers also reported 



that time for discussion and sharing of ideas 
with county officials and colleagues was in- 
valuable. 

The principal's role was critical in reducing 
tension. Principals who were actively involved 
with the instructional changes created environ- 
ments in which common goals were established 
and creativity and risk-taking were encouraged. 
Teachers felt supported in their efforts. Given 
the enormous financial burden placed on local 
education agencies, principals were also critical 
in schools' efforts to secure external grant 
funding. This additional funding provided 
schools with some of the materials needed for 
achieving change that otherwise would have 
been unavailable to them. 

By teachers' accounts, the instructional 
changes brought about by the MSPAP have 
had positive effects on students' motivation for 
reading and writing. Additionally, teachers 
reported positive effects on students' attitudes 
toward literacy and on their academic behav- 
ior. Although the pressures of altering their 
curricu' m produced frustration and anxiety, 
teachers were pleased with the changes being 
made and felt that the changes were worth- 
while, given the positive effects on students. 
Similarly, they reported that although planning 
for instruction was now more time-intensive, 
they enjoyed teaching more. Thus, it seems 
that based on the evidence in interview com- 
ments and state documents, positive instruc- 
tional and affective changes have resulted from 
the introduction of the MSPAP as a statewide 
accountability instrument and these changes are 
consistent with the goals of the statewide pro- 
gram. 
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CONCERNS AND FUTURE RESEARCH 
DIRECTIONS 

Many concerns should be considered when 
implementing a statewide performance assess- 
ment such as the MSPAP. One is the problem 
of holding students responsible for knowledge 
that they may not have had a fair chance to 
acquire, given the inequitable circumstances of 
various schools and districts (Porter, 1993). 
However, according to "The Report on the 
Governor's Commission on School Perfor- 
mance" (Sondheim et aL, 1989), the ultimate 
goal of the school accreditation aspect of the 
MSPP is to identify schools that are unable to 
provide the necessary services for instructing 
their students and give assistance at the state 
level. 

Concern about such state-mandated action 
feeds the argument that the demand for such 
curricular reforms places too much weight on 
accountability to achieve desired educational 
reform. Similarly, some have contended that 
such curricular reforms, in their efforts to 
bring quality education to all students, employ 
governmental influence and coercion as a 
means of achieving change at the expense of 
democracy (Capper & Jamison, 1993; Shann- 
on, 1991, 1993). Mehrens (1992) has also 
warned that if performance assessments are 
used for accountability purposes, as in Mary- 
land, they may face many of the same prob- 
lems that traditional norm-referenced, stan- 
dardized tests have faced. All of these are valid 
issues that warrant consideration when inter- 
preting the results of any investigation of the 
consequences of a state-mandated assessment 



on instructional practice or when making 
policy and instructional decisions. 

Although this investigation provided sub- 
stantial support for the premise that instruction- 
al change has resulted from the MSPAP, the 
results are based primarily upon interview 
comments from teachers and administrators. 
More intensive studies that document instruc- 
tional changes and link them to indicators are 
needed. Such investigations would provide 
converging sources of evidence from observa- 
tions, field notes, videotapes, and interviews 
with teachers, administrators, and students 
regarding the types of changes and the effects 
of such changes that result from high-stakes 
performance assessments. 
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APPENDIX 
Time Line of Data Collection Procedures 



April 17, 1992 Initial contact with Dr. Robert Gabrys, Assistant Superintendent 

for School Performance, Maryland State Department of Education 
(MSDE) gaining permission to access information collected and 
analyzed by MSDE^ regarding the Maryland School Performance 
Assessment Program (MSPAP). 

May 1992 Exploratory phase of the research: 

• Drafting of teacher interview questions 

• Classroom observations of the MSPAP being administered 

• Piloting of teacher interview questions and procedures 

June 1992 Transcription and coding of pilot interviews. 

Revision of interview questions based on exploratory research. 

June 25, 1992 Initial contact with Assistant Superintendents of Instruction in each 

of the 24 Maryland public school systems requesting their assis- 
tance in identifying one or more representatives from their school 
district who were knowledgeable about the ways reading instruc- 
tion in the elementary schools may have changed as a result of the 
MSPAP. 



August 27- County Supervisors from 21 of the 24 Maryland public school 

October 29, 1992 systems interviewed in order to describe how instruction in their 

county has changed as a result of the MSPAP and in order to 
nominate elementary schools within their district that were making 
changes in response to the MSPAP attempted in each school 
nominated categorized. Five schools selected for inclusion in 
school-based interviews. 



November 1992 County interview data condensed and types of innovations attempt- 
ed in each school categorized. Five schools selected for inclusion 
in school-based interviews. 
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November 12, 1992- 
January 19, 1993 



January 7, 1993 

8:30- 9:15 

9:15-10:00 
10:00-11:00 
11:00-11:30 
12:00-12:30 

January 14, 1993 
9:30-10:30 

January 25, 1993 
10:00-10:45 
10:45-11:30 
11:30-12:15 

February 4, 1993 
8:45- 9:15 
9:30-10:30 
10:30-11:30 
11:30-12:30 
12:30- 1:30 

February 25, 1993 
10:00-10:30 
10:30-11:15 
11:15-12:00 
12:00-12:15 
12:45- 1:15 

March 26, 1993 
10:00-10:45 
10:45-11:15 
11:15-12:00 



County Board of Education offices and five research sites contacted 
in order to gain permission to conduct school-based interviews with 
building administrators, reading specialists, and third- and fifth- 
grade teachers in each of the five elementary schools. 

Data collection at School E, interviewed: 
Former Cumcuium Coordinator of school 
Third-grade teachers (3 teachers) 
Fifth-grade teachers (5 teachers) 
Current Curriculum Coordinator of school 
Reading Specialist in school 

Data collection at School E, interviewed: 
Principal of school 

Data collection at School A, interviewed: 
Principal of school 

Third-grade teachers (2 teachers and Gifted & Talented teacher) 
Fifth-grade teachers (2 teachers) 

Data collection at School B, interviewed: 
Assistant principal of school 
Fifth-grade teachers (3 teachers) 
Third-grade teachers (2 teachers) 
Reading Specialist in school 
Principal of school 

Data collection at School D, interviewed: 
Reading Specialist in school 
Fifth-grade teachers (4 teachers) 
Third-grade teachers (6 teachers) 
Supervisor of Instruction 
Principal of school 

Data collection at School C, interviewed: 
Third-grade teachers (2 teachers) 
Fifth-grade teachers (2 teachers) 
Principal of school 
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